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Abstract: A life cycle metaphor characterizes the evolving 
relationship between the evaluator and program staff This 
framework suggests that common developmental dynamics 
occur in roughly the same order across groups and settings. 
There are stage-specific dynamics that begin with Pre- 
History, which characterize the relationship between the 
grantees and evaluator. The stages are: (a) Pre-History, 

(b) Process, (c) Development, (d) Action, (e) Findings/ 
Compilation, and (f) Transition. The common dynamics, 
expectations, and activities for each stage are discussed. 


Life cycle metaphors have been used to explain the developmental 
process of task groups, treatment groups, and organizations (Cameron & 
Whetten, 1983; Hasenfeld & Schmid, 1989). Underlying this framework is 
the assumption that these groups move through developmental stages that 
are roughly akin to biological processes. This framework suggests that the 
same developmental dynamics occur in the same order across differing groups 
and settings. 

The Circles of Care (CoC) experience suggests that the evaluation 
process can be conceptualized as life cycle. We have observed stage-specific 
dynamics that characterize the relationship between the CoC grantees and 
the evaluator. There are distinct developmental levels that help to explain 
the nature and content of each party's actions and concerns. Given that the 
evaluator's engagement in the CoC process is participatory (e.g., he or she 
is a critical member of the planning team), understanding the relational 
bond that is gradually formed and the stage- specific tasks that are undertaken 
can help future planners and evaluators to assess the process by which they 
and their projects develop (Rubin & Babbie, 1993). This understanding may 
also help groups to review their developmental pace and to troubleshoot 
common stage-specific issues that arise throughout the process. 
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Understanding the evaluator's relationship to the CoC initiative also 
necessitates consideration of the members' pre-history, which refers to their 
history before the creation of the setting (Sarason, 1971; 1996). There may 
be unique dynamics that distinguish the evaluation life cycle from other 
endeavors, based on historical experiences of the American I ndian/Alaska 
Native (AI/AN) and the non-AI/AN status of many CoC evaluators. 

The CoC initiative was born out of a history of conflict between Al/ 
AN communities and federal and state governments, both broadly and 
specifically surrounding American I ndian child welfare and health issues (for 
a thorough review of this history, see Freeman et al.'s paper in this Special 
Issue). These encounters have often been highly emotional and negative 
experiences for AI/AN communities which culminated with an important series 
of legislative innovations: the Indian Self-Determination and Education 
Assistance Act (1975), the Indian Health Care Improvement Act (1976), and 
Indian Child Welfare Act in (1978). 

Furthermore, the experiences of AI/AN communities with regard to 
research, including evaluation research, reveals a history of practices and 
approaches that have alienated not only individual AI/AN people but entire 
AI/AN communities (Darou, Hum, & Kurtness, 1993; Manson, 1989; Norton 
& Manson, 1996). This history, which predated the creation of CoC, is thus 
important in formulating our understanding of the CoC planning process and 
its evaluation. It required CoC planners to initiate a process of discussing 
their various ideas and preferred agendas for the evaluation with a perceived 
y'udgmenta/outsider (i.e., the evaluator), and in addition, required working 
through the various tensions that were associated with this new relationship. 
In the end, this process created a shared sense of co-membership and a 
common base among the participants that allowed culturally diverse groups 
of AI/AN and non-AI/AN people to reflect upon their shared values, and to 
rise above a prehistory of conflict (Erickson, 1975; Hornby, 1993). 

The Evaluation Life Cycle 

With this pre-history as context, six developmental life cycle stages 
can be distinguished as separate and distinct evaluation phases. They are: 
(a) the Pre-history Stage, (b) the Process Stage, (c) the Development Stage, 
(d) the Action Stage, (e) the Findings/Compilation Stage, and (f) the Transition 
Stage. The common dynamics, expectations, and activities for each of these 
stages are discussed below, followed by concluding remarks intended to 
summarize the evaluation life cycle. 


Pre- History Stage 

This stage is characterized by relationship testing and building. I n 
many instances, this is the first time that an outside contract evaluator has 
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been engaged to assess a particular agency-based AI/AN program. The 
notion of a non-AI/AN person involved in the evaluation, though a contract 
employee, can conjure feelings among the CoC program staff of prior adverse 
experiences within both professional and personal contexts. I n many instances, 
the evaluator is unknown to the program staff prior to his or her engagement, 
having been recruited from a local university's psychology, sociology, or social 
work department, or vis-a-vis a recommendation provided by an allied service 
provider. In either instance, the evaluator and the program staff are 
unacquainted with one another, and each may have preconceived notions 
about the other. 

The program staff may view the evaluator as an outsider, someone 
who is not part of the CoC team. I n fact, the evaluator may be suspect for 
several reasons that include: (a) non-AI/AN heritage; (b) socio-demographic 
roots that differ from the local AI/AN community; (c) academic and research 
interests that may be perceived to be out of sync with the project's 
programmatic focus; and (d) the nature of the engagement itself, whereby 
the evaluator is compensated, as is the staff, and yet there is a perception 
that the evaluator's motivation is not derived from the same selfless 
commitment to helping the AI/AN community as is that of the program staff. 

There is also distrust of the evaluator's judgment: that he or she 
will inaccurately interpret and report process and outcome domains, or 
demonstrate the insensitive qualities that all-to-often have been displayed 
by those charged with assessing AI/ANs. These feelings on the part of 
program staff often confront the evaluator as s/he enters the agency for the 
first time. 

Similarly, the evaluator may have preconceptions that affect the 
work relationship. The evaluator may be concerned that he or she will not 
understand the specific context of Al /AN processes, needs, and services that 
will be evaluated. The evaluator may also be concerned s/he will not display 
or fully comprehend the cultural nuances that are an essential part of the 
assessment despite her/his technical competence. This latter point is 
especially critical as cultural context is an essential component of the 
evaluation process, affecting the form and substance of the assessment 
inquiry and influencing its interpretation. 

Conversely, the evaluator may incorrectly assume knowledge or 
comprehension of ethnic and racial groups from prior experiences that do 
not conform to those of the AI/AN community. Spiritual awareness, non- 
verbal communication, talking circles, and reverence for elders are often 
part and parcel of the planning design and program implementation process. 
These cultural-specific dynamics can be confounding for the uninitiated 
evaluator, and consequently, may cause misrepresentation of events. 

Furthermore, differing work approaches, reliance on a technical 
vocabulary and conflicting evaluation paradigms can create barriers between 
the evaluator and program staff. Some evaluators are not accustomed to 
working as part of a team, and, if they do participate, their involvement is 
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relegated to their expertise in evaluation. The evaluation design process 
can sometimes be a solitary one, where drafts are presented and input 
sought, with communication bi-directionally alternating (from program staff 
to evaluator and back again) without frequent face-to-face interactions. By 
contrast, AI/AN members of the team come from a tradition of conjoint 
planning and active participation that influence all facets of the CoC planning 
process. Thus, the expectation of the AI/AN participants is one of ongoing 
dialogue with the evaluator, their participation in the design of the process, 
data collection and interpretation. 

The evaluator's technical vocabulary is also full of research jargon 
and methodological requirements that are little understood and perhaps even 
less trusted by program staff. Similar to the perception that federal, state 
and local laws are sometimes misapplied to AI/ANs is the concern among 
many AI/AN members of the program staff and community that the process 
of evaluation is not relevant or applicable. The evaluator may be seen as 
representing the values and beliefs of "the establishment" (i.e., the 
mainstream) that are incompatible with AI/AN ways. The program staff 
may grapple with the question, "Whose side is the evaluator on?" as they 
describe the importance of preparing an I nstitutional Review Board application, 
to be reviewed by an all-White panel of university or I ndian Health Service 
researchers. 

Since CoC requires an assessment of the planning process, an early 
decision must be made as to whether the evaluator's role is that of "participant 
observer" (Rubin & Babbie, 1993; Rossi & Freeman, 1993) or the more 
conventional "sidelines- objective" observer. For many evaluators, the latter 
is the more familiar, requiring a lesser level of engagement. Yet this approach, 
though methodologically valid, fulfills Pre-History notions of the evaluator as 
an outsider. The evaluator is seen as not being truly part of the team, but 
rather as a critical, unforgiving assessor of all that is deficient and that requires 
intervention. Though not congruent with the evaluator's intentions, these 
perceptions argue for participant observation, enabling the evaluator to be 
seen as a fully participating member of the team. 

I n light of these and other possible issues that may arise early in the 
process, discussion between CoC program staff and the evaluator concerning 
similarities and differences in style, approach, and expectations is encouraged 
at the Pre-History stage. Overall, a stronger foundation for the entire 
evaluation process can be built with the open expression and 
acknowledgement of these personal feelings and concerns. The three-year 
working relationship between program staff and evaluator is dependent on 
each party's trust of the other as well as commitment to working together as 
one united community enterprise. Though not instantly created, good will, 
acknowledgements of differences, and openness to learning, can each 
contribute to a solid beginning. 



34 


VOLUME 11, NUMBER 2 


Process Stage 

The work of evaluation truly begins in the Process Stage. During 
this stage, the evaluation process itself is defined. This definition includes 
how (a) the evaluation will relate to planning activities, (b) provide data to 
fulfill Center for Mental Health Services (CMHS) grant requirements, and (c) 
fuel the planning process itself. Program staff will likely feel somewhat 
unsure at this stage. That is, although they may have strong community- 
based program skills, they likely do not have formal planning experience and 
may have even less exposure to the process of evaluation. 

The evaluator is still seen as a new, somewhat tangential, member 
of the group. Though the program staff may have worked together before, 
this is the first time that program staff and evaluator will meet to discuss 
project activities. Each may be silently questioning the process, as outlined 
in the following hypothetical questions. 

Program staff : How will we work together? How much 
explaining about my ideas and recommendations do I need 
to make? Should I ask for input from the evaluator, or am I 
asking for trouble? We haven't discussed the resource needs 
for the assessment process. I s it my responsibility, or that 
of the evaluator, to direct this component? 


Evaluator : How will we work together? How much 
explaining about my ideas and recommendations do I need 
to make? What do they expect of me? If I have an idea 
about their program, should I offer it or will I be considered 
ill-informed and pretentious? Is conducting the resource 
needs assessment, which relies on statistical compilation, 
a part of my responsibilities? 

As is evidenced by these questions, getting to know one another 
and clarifying roles and expectations is an integral part of the Process Stage. 
While there may have been general philosophical discussions at the Pre- 
History Stage about how issues are approached and tasks undertaken, it is 
in the Process Stage that true interaction begins. This is the point at which 
the program staff and the evaluator begin to express themselves on 
substantive issues, where reasoning is explained, and where all begin to 
learn and to teach each other. 

While program staff begin to determine an approach, mapping the 
community and its resources, as well as identifying potentially supportive 
and restraining forces, the evaluator undertakes the observation of the 
planning process. Given the newness of the evaluator's involvement, there 
may be a feeling of self-consciousness on the part of staff and evaluator 
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alike. Though participant-observation is a proven method for assessing the 
planning process, the likely perception among staff that the evaluator is not 
truly an integral part of the program inherently suggests that her/his 
participation is not fully accepted. 

An important component of the Process Stage is the resource needs 
assessment. This assessment is a joint endeavor between program staff 
and the evaluator. It is an opportunity to meld the knowledges of program 
staff and evaluator and to test the developing relationship among its 
participants. Program staff members are most attuned to the community 
and its resources. They are familiar with the major institutions in the 
community, the formal and informal systems of care that exist, and where 
initial support and resistance to the CoC initiative may be present. Staff 
members are also responsible for collecting the needs assessment data. 
This includes requesting and cajoling health, human services, and educational 
personnel to provide detailed information on their services, programs, and 
beneficiaries. 

The evaluator's role in the resource needs assessment is that of 
managing the data that is received. While some data are shared in raw 
form with the Circles of Care Technical Assistance Center (CoCETAC), the 
information also forms the foundation for community-based planning activities 
and for baseline assessment of the program's progress. Thus, the evaluator 
is responsible for the important task of organizing the needs assessment 
data into manageable and meaningful components. 

At this point there is a convergence of interest. Program staff and 
evaluator alike are interested in the results of the needs assessment. The 
program staff have specific questions that they would like to have answered 
so they can confirm or disconfirm their knowledge of community needs based 
on the information that has been reported. For example, they may wish to 
know how many AI/AN youth are involved with the county's probation 
department? Or, what is the ratio of foster care placements by the child 
welfare agency for Al /AN youth relative to other youth in the service area? 
The evaluator is similarly interested in these and other questions, as they 
represent baseline information that is descriptive of the community that can 
be subsequently reassessed to determine whether positive change has 
occurred. This capacity to demonstrate the impacts of the CoC intervention 
is also of interest to the program staff. 


Development Stage 

Having successfully engineered the Process Stage, the program staff 
and evaluator proceed to the Development Stage. 1 1 is at this stage that the 
partnership between program staff and evaluator is cemented. Their initial 
work together during the Process Stage makes it easier to proceed into new 
domains of less prescriptive assessment and evaluation. Virtually every 
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planning meeting or programmatic undertaking (e.g., community picnic, 
newsletter. Gathering of Native Americans - GONA event) should have an 
associated evaluation component. I n some instances the evaluation is part 
and parcel of the program's design, while at other times the evaluator's role 
as participant observer is most applicable. I n either situation, however, trust 
and communication between staff and evaluator are essential. 

Long-term perspectives on the planning process characterize the 
Development Stage. A master calendar of events and meetings is often 
established, whereby program staff and evaluator discuss incremental activities 
and timelines. While program staff focus on the preparation of materials, 
acquisition of event resources, and garnering support for specific events, the 
evaluator probes staff for relevant measures and consistently notes aspects 
of the planning process that may be informative to the overall evaluation 
process. 

While program staff may have ideas about what measures are 
relevant for each undertaking, they will likely look to the evaluator for direction 
on specific instruments and their applicability. Whether it is a question of 
satisfaction with a planning meeting or the impact that a GONA or community 
picnic has had on participants' views, the evaluator is charged with developing 
questions and identifying or creating scales that will capture this information. 
Since a longitudinal perspective is available vis-a-vis the three-year planning 
process, care is taken to select measures that can be tested and retested 
over time. Changes in attitude and beliefs as well as knowledge about 
resources and regional AI/AN history, can be assessed and compared at 
periodic intervals. 

A sample of standardized measures, including an explanation of 
instrument validity, can be obtained from the evaluator at this stage. However, 
determination as to the utility of each measure from a planning and program 
development perspective rests with the program staff. It should become 
clear at this stage that the evaluator and the evaluation process are in the 
service of the program, and that the program does not exist to serve the 
evaluation. From this perspective, each party's role and function on the 
team is clearly defined. Indeed, it is during the Development Stage that 
roles and responsibilities are clearly understood. 

Consequently, though there may have been an orientation to the 
role of the evaluator at the Pre-History or Process Stage, this role bears 
repeating at the Development Stage. 1 1 is likely that with a clearer sense of 
their role and subsequent inclusion as a member of the team, the evaluator's 
contribution will be better understood, valued, and utilized. This experience 
may also prove to be cathartic in that early assumptions by program staff 
about the evaluation process and the evaluator can be explored and redefined 
in light of current roles and responsibilities. At the Development Stage, the 
CoC planning process is reaching its stride, and will be fully realized in the 
Action Stage. 
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Action Stage 

Building on the Development Stage's solid planning, it is during the 
Action Stage that program staff and evaluator give meaning to the word 
"team." They are working in unison. J ust as a GONA event requires meal 
planning, program planning, and recreational/social planning, so too does it 
require evaluation planning. And, just as members of the program staff 
assume responsibility for the first three activities, the evaluator as a team 
member is responsible for the latter component. Distinctions are no longer 
made between evaluation and program. Rather, it is now viewed as one 
process with several inter-related parts. 

The collaboration among team members that characterizes the action 
stage is enhanced by the introduction of emerging evaluation findings. Though 
the evaluator is schooled in data coding, data entry, data manipulation, and 
data presentation, the interpretation of findings and the cross-tabulation of 
results based on planning assumptions and program-related questions come 
from the program staff. I n line with the evaluator's role of furthering the 
planning process, findings need to be driven by planning and programmatic 
concerns. In other words, preliminary findings, which fuel a process of 
continued evaluation, are used to enrich program and planning activities. 

Process evaluation also takes on new meaning during the Action 
Stage. Though process evaluation is an attempt to chronicle planning 
activities, seeking out distinct phases and benchmarking events that inform 
others who may attempt to replicate the planning effort, it also serves as a 
staff development tool. The evaluator is in a unique position to ask questions 
about the meaning of certain activities, and to query staff about their priorities 
and concerns. The dialogue that ensues between evaluator and program 
staff helps the latter to reflect on their immediate situation, to address specific 
issues that the evaluator has helped them to frame, and to view the planning 
process itself as a replicable and dynamic vehicle for producing change. 

A shift has thus occurred in the relationship between program staff 
and evaluator. At an earlier stage the program staff likely perceived evaluation 
as an additional requirement, a non-negotiable add-on to contend with in 
the interest of receiving a CoC grant. However, by the Action Stage, evaluation 
is now an integral part of the overall effort, as is the evaluator. It is also 
during the Action Stage that the fruits of the planning effort begin to be seen, 
progress acknowledged, and an end-direction perceived. 


Findings/ Compiiation Stage 

CoC activities are in high gear during the Findings/Compilation Stage. 
Findings from the community engagement and evaluation process are 
contributing to policy and planning recommendations, and reports are being 
prepared to satisfy grant requirements. An abundance of primary and 
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secondary information is being obtained, catalogued, and interpreted. 
Program staff and the evaluator are interested in better understanding the 
meaning of the data that they have compiled, while also testing their tentative 
conclusions. 

Meetings are convened with key informants and community members 
to review data and to confirm or disconfirm interpretations of findings. 
Program staff and the evaluator explore the meaning of the information 
obtained with these participative, community-based respondents. Possible 
programmatic and systemic solutions are discussed. In some instances, 
new and additional insights emerge from these sessions, as respondents 
explain subtleties in the data, and steps are considered that will transition 
CoC from planning to implementation. 

Gaps in information also emerge at this stage. Program staff and 
the evaluator must decide whether additional surveys or interviews are 
necessary. Like earlier stage activities, survey or interview questions will be 
jointly developed by the team, with the evaluator responsible for assuring 
that the information obtained can be presented in a way that informs the 
planning process. 

By the Findings/Compilation Stage, much has been written about 
aspects of the initiative that need to be compiled into a comprehensive report 
with recommendations. Though the evaluator or another team member 
with strong writing skills will ultimately pull the narrative together, 
conceptualization of the plan's elements is the responsibility of the entire 
team. If the evaluator is not the writer, the presentation of data is reviewed 
by the evaluator to assure accuracy. 

As the implementation plan is developed, the program staff begin to 
explore funding opportunities that will support the introduction of services. 
The justification of need, an integral part of funded proposals, requires 
statistical and other proofs that the project is rooted in a clear understanding 
of the community and the intended beneficiaries of services. The abundance 
of evaluation data that has been amassed can be used to support this section 
of the proposal. I n addition, should a separate evaluation be required as a 
condition of funding, the evaluator, who is familiar with the project plan, can 
be called upon to develop the evaluation design and possibly oversee its 
execution. 

At the close of the Findings/Compilation Stage, a clear plan emerges 
with strategies and resource considerations that can lead to implementation. 
The seemingly unlikely marriage between program and evaluation, while 
consummated at the Action Stage, can now point to the progeny of its union: 
the creation of a developmental plan with specific recommendations for 
implementation. And, like doting parents, there is pride in the offspring and 
its potential, which begins to be realized at the Transition Stage. 
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Transition Stage 

During the Transition Stage, the evaluator and program staff are at 
another juncture in the project's development and their evolving relationship. 
A choice point is reached as the planning phase ends and implementation 
begins. Will there be a role for the evaluator as the project moves from 
planning to implementation? Are resources sufficient to support this transition? 
If funding is limited what value is placed on the evaluation process? This 
question becomes particularly relevant when the funder does not require an 
independent assessment. 

The Transition Stage thus may represent an end to the evaluation 
process through the preparation of the final report, the formal separation of 
the program staff and evaluator, and the contract termination. Alternatively, 
it may represent the beginning of a new work relationship, as the project 
moves toward implementation. There is also the possibility that individual 
program staff members or the evaluator may not be available beyond their 
initial three-year CoC commitment. Consequently, the project and its plan 
for evaluation, which are universally understood as an important part of 
program services, may continue with a revised agenda or a changed 
composition of program team members, each bringing to their new career 
assignments an enhanced understanding of roles and how the performance 
of their duties is dependent upon the expertise of other team members. 

Even if a new team is assembled to carry on with the project, the 
continuity between planning and implementation is essential. Some would 
argue that these are different sides of the same coin, meaning that there is 
no currency unless planning and evaluation are united. And so, unlike the 
Pre-History Stage that began the evaluation process, at this transition it is 
likely that an evaluation component will be carried forward as an expected 
component of the I mplementation Phase. 

Several data elements on which the CoC evaluation rests also are 
important to the Transition Stage. They represent baseline information on 
which the impact of one or more interventions will be assessed. Evaluating 
complex data uncovered during the planning phase and subsequently 
designing interventions that address these issues are a fundamental and 
necessary part of this transition. Both program staff and evaluator at this 
stage are aware of the interdependence between the two phases, and having 
immersed themselves in an in-depth understanding of the problems facing 
the community, they are keenly interested in producing positive change. 

Conclusion 

With the Transition Stage complete, the Evaluation Life Cycle is ended 
and program implementation begins. Both parties now understand and 
appreciate the relationship between program staff and evaluator, which was 
once tentative. The success that characterizes the end of the evaluation 
process will likely carry forward into program implementation. 
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We have seen the nature of the work and the work relationships 
progressively change from the Pre-History Stage, to the Process Stage, to 
the sequential stages of Development, Action, Findings/Compilation, and 
Transition. The collective energies of the program staff and the evaluator 
have produced a series of outcome reports and a plan that addresses the 
unique mental health needs of AI/AN youths and their families. The three- 
year timeframe has afforded the planning process an opportunity to develop 
and mature across the six stages. 

Though each stage has been described by key characteristics, 
including the evolving relationship between program staff and the evaluator, 
both the length of time that comprises each stage, and the confidence that 
all stages will be attained during the planning cycle, are uncertain. The 
planning and evaluation processes for each of the grantees encompass a 
different set of circumstances, and consequently, differing experiences and 
outcomes. 


Gary Bess, Ph.D. 
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